Measuring content consumption

ABSTRACT

Techniques to measure consumption of content pages comprising a plurality of distinct content assets are disclosed. In various embodiments, content consumption signal data gathered by a plurality of clients, the content consumption signal data reflecting for at least a subset of content pages user engagement by content asset comprising the content page, is received. The received content consumption signal data and content attribute data associated with each respective content page are used to compute for each content page a content consumption metric reflecting an amount of content determined to have been consumed.

CROSS REFERENCE TO OTHER APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/880,278, entitled MEASURING CONTENT CONSUMPTION filed Jan. 25, 2018which is incorporated herein by reference for all purposes, which is acontinuation of U.S. patent application Ser. No. 15/142,439, entitledMEASURING CONTENT CONSUMPTION filed Apr. 29, 2016, now U.S. Pat. No.9,912,768, which is incorporated herein by reference for all purposes,which claims priority to U.S. Provisional Application No. 62/154,919,entitled MEASURING CONTENT CONSUMPTION filed Apr. 30, 2015 which isincorporated herein by reference for all purposes.

BACKGROUND OF THE INVENTION

Providers of content, such as publishers of web pages or other contentpages, native advertisers, or other providers of content pages, may wantto know how much of the content they are providing is being consumed,e.g., by a typical content consumer, specific types of consumer, etc.,and/or how effective the content has been in achieving a desiredbehavior, such as click through to further content, completing apurchase, etc. Tools exist to detect whether such desired behavioroccurred in response to a display page or ad, but a publisher may desireto know which specific content assets drove such behavior.

Native advertising refers to displaying ads or other sponsored contentin a manner that integrates such content with other, non-sponsoredcontent in a manner that matches a native look and feel of the page orother display in which the native advertising content is included. Forexample, a newsfeed style of page or display may include nativeadvertising content interspersed, e.g., in a prescribed way, among othercontent items presented in the newsfeed. Native advertising content maylink to other, more in depth content, such as an article formattedand/or otherwise presented in a manner that is consistent with a styleassociated with a publisher content page in which the native ad wasdisplayed.

Native advertising requires the distribution of content at scale intomultiple publisher sites. In various embodiments, native advertisingcontent may comprise “brand” content and can consist of a variety ofeditorial components including words, images, video, sound or anycombination thereof. This content is used as advertising messaging.

Native advertising and other content pages may comprise multipledistinct content assets, such as images, paragraph-formatted text,bulleted lists, product comparison charts, user-posted comment sections,etc. An article (or other unit of native advertising or other content)may include multiple types of content assets. Content may include video,images, text, sub-headlines, captions, slide shows (collection ofimages), animated images such as GIFs, informational images(infographics), interactive elements, and embedded social content suchas Twitter™ “tweets” or other social network posts.

Publishers, advertisers, and other may wish to know how much value eachasset comprising a content page is contributing to an intended purposeof the page, such as to advertise a given product. So-called “heat maps”have been provided to indicate the portions of a page which users spentthe most time viewing, however to date such heat maps have been oflimited utility, typically merely confirming that more user time isspent viewing the top portion of a page, which may be displayed first,than portions that may require users to scroll down.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of the invention are disclosed in the followingdetailed description and the accompanying drawings.

FIG. 1 is a block diagram illustrating an embodiment of a contentconsumption measurement system and environment.

FIG. 2 is a block diagram illustrating an embodiment of a contentconsumption measurement system.

FIG. 3 is a flow chart illustrating an embodiment of a process tomeasure content consumption.

FIG. 4 is a flow chart illustrating an embodiment of a process todetermine content attributes related to measuring content consumption.

FIG. 5 is a flow chart illustrating an embodiment of a process todetermine content attributes.

FIG. 6 is a flow chart illustrating an embodiment of a process toinstrument content pages to measure content consumption.

FIG. 7 is a flow chart illustrating an embodiment of a process tomeasure and report content consumption.

FIG. 8 is a diagram illustrating an example of a content consumption“map” display in an embodiment of a content consumption measurementsystem.

FIG. 9 is a flow chart illustrating an embodiment of a process tomeasure content consumption based at least in part on individual userattributes.

FIG. 10 is a flow chart illustrating an embodiment of a process toadjust content consumption measurement based on observed user behavior.

FIG. 11 is a flow chart illustrating an embodiment of a process tomeasure and report content consumption across content sources and types.

FIG. 12 is a flow chart illustrating an embodiment of a process tomeasure consumption based on content value.

DETAILED DESCRIPTION

The invention can be implemented in numerous ways, including as aprocess; an apparatus; a system; a composition of matter; a computerprogram product embodied on a computer readable storage medium; and/or aprocessor, such as a processor configured to execute instructions storedon and/or provided by a memory coupled to the processor. In thisspecification, these implementations, or any other form that theinvention may take, may be referred to as techniques. In general, theorder of the steps of disclosed processes may be altered within thescope of the invention. Unless stated otherwise, a component such as aprocessor or a memory described as being configured to perform a taskmay be implemented as a general component that is temporarily configuredto perform the task at a given time or a specific component that ismanufactured to perform the task. As used herein, the term ‘processor’refers to one or more devices, circuits, and/or processing coresconfigured to process data, such as computer program instructions.

A detailed description of one or more embodiments of the invention isprovided below along with accompanying figures that illustrate theprinciples of the invention. The invention is described in connectionwith such embodiments, but the invention is not limited to anyembodiment. The scope of the invention is limited only by the claims andthe invention encompasses numerous alternatives, modifications andequivalents. Numerous specific details are set forth in the followingdescription in order to provide a thorough understanding of theinvention. These details are provided for the purpose of example and theinvention may be practiced according to the claims without some or allof these specific details. For the purpose of clarity, technicalmaterial that is known in the technical fields related to the inventionhas not been described in detail so that the invention is notunnecessarily obscured.

Techniques to measure content consumption and/or performance aredisclosed. In various embodiments, techniques disclosed herein mayenable advertisers to understand how much of their content was consumedby the end user and/or which content assets comprising a page have beenconsumed. For example, how much of an article was read? How much of avideo was viewed? Did the user scroll down far enough to see the image?How much of the image was on screen, and for how long? Etc.

In various embodiments, content pages may be pre-processed to ascribe toeach of at least a subset of content assets comprising the page acorresponding proportion of the content value of the page. The relativevalue of respective content assets may be determined based on criteriasuch as size, placement within the page, word or other content density,complexity, semantic content, formatting, etc., and may be discounted byfactors that may increase user viewing time but may not be (as)associated with a primary purpose of the content.

In various embodiments, content consumption and/or performance may betracked by content asset, user, user type, etc. Performance may bemeasured, by content asset, based at least in part on post-consumptionactivity, e.g., the extent to which users who “consumed” a certaincontent asset engaged in a desired behavior (immediately, subsequent toother actions, and if so which actions, etc.), such as clicking throughto more detailed content, taking action called for by such detailedcontent, making a purchase of an advertised product, etc.

In various embodiments, content consumption and/or performancedetermined as disclosed herein may be used to guide the revision of acontent page, such as to make more prominent a content asset that hasperformed well, or to increase the visibility of an important asset thatmay be underperforming due to placement in the page, and/or to guide thepreparation of future pages.

FIG. 1 is a block diagram illustrating an embodiment of a contentconsumption measurement system and environment. In the example shown,content consumption measurement system and environment 100 includes aplurality of client devices, represented in FIG. 1 by client devices102, 104, and 106, have access via the Internet 108 to content pagesdownloadable from a plurality of web servers, represented in FIG. 1 byweb servers 110, 112, and 114, each of which in the example shown servesweb pages from an associated web page data store 116, 118, and 120,respectively.

In the example shown, a content consumption and performance analysisserver and/or other system (e.g., a group of servers; agents running onone or more of client devices 102, 104, 106 and/or web servers 110, 112,114, etc.) 122 is connected to the Internet 108. In various embodiments,content consumption and performance analysis server 122 may downloadfrom web servers (e.g., 110, 112, 114) and analyze web pages (e.g.,stored in 116, 118, 120) to determine and store in content consumptionand performance database 124 one or more of content page structure(e.g., page DOM or other structure information) information and contentpage attribute information, such as an identification of one or morecontent assets comprising the page and for each one or more attributessuch as content asset type, content complexity, and/or other attributesassociated with one or both of content consumption metrics and contentperformance.

FIG. 2 is a block diagram illustrating an embodiment of a contentconsumption measurement system. In various embodiments, the contentconsumption measurement system shown in FIG. 2 may be used to implementcontent consumption and performance analysis server 122 of FIG. 1. Inthe example shown, content consumption and performance analysis server122 of FIG. 1 includes a network communication interface 202, such as anetwork interface card and associated software components, to provideconnectivity to other nodes via the Internet and/or one or more othernetworks. Content page attribute analysis module 204, which may comprisea functional module provided via software running on a CPU or otherprocessor comprising content consumption and performance analysis server122 (not shown in FIG. 2), retrieves content pages, such as web pages, aprocesses the pages to determine the structure and/or other content pageattributes of such pages, and to store such attributes in a content pageattributes database 206.

At content page display time, a consumption signal receipt and storagemodule 208, which may comprise a functional module provided via softwarerunning on a CPU or other processor comprising content consumption andperformance analysis server 122, receives via communication interface202 content consumption signal data from client systems/devices at whichcontent pages have been displayed. For example, code embodied in and/orcalled by such content pages may cause consumption signal data to begathered at the client and reported to which may comprise a functionalmodule provided via software running on a CPU or other processorcomprising content consumption and performance analysis server 122.Examples of consumption signal information may include, withoutlimitation, one or more of the following: time page displayed; time eachrespective defined portion, e.g., each content asset comprising thepage, was displayed; mouse or other cursor movement and/or clicktracking data; scroll data; user agent identification and/or attributedata (e.g., browser used, display size and/or form factor, etc.); othertiles displayed concurrently with content page, and size and location ofeach relative to content page; etc. In the example shown, consumptionsignal data may be stored in a memory device 210 and/or other storage.

In the example shown, consumption and performance measurement module212, which may comprise a functional module provided via softwarerunning on a CPU or other processor comprising content consumption andperformance analysis server 122, uses content page attribute data (204,206) and consumption signal information (208, 210) to compute contentconsumption and performance metrics for individual users and/or toaggregate such information across users, content pages, and/or contentdelivery media, devices, networks, and/or other domains. Consumption andperformance module 212 stores consumption and/or performance metric datain a data, such as database 124 of FIG. 1, via a database interface 214.Consumption and/or performance metrics and/or analyses are analyzed,aggregated, and reported by a reporting module 216, which in variousembodiments may comprise a functional module provided via softwarerunning on a CPU or other processor comprising content consumption andperformance analysis server 122.

FIG. 3 is a flow chart illustrating an embodiment of a process tomeasure content consumption. In various embodiments, the process of FIG.3 may be implemented by a content consumption and performance analysissystem, such as content consumption and performance analysis server 122of FIG. 1. In the example shown, web (or other content) pages areprocessed to determine and store page structure and/or content attributedata (302). Content pages are configured, e.g., by including a snippetof code, to cause content consumption signal data to be gathered andreported (304). Consumption signal data is received from client devicesat which the content page(s) is/are displayed, and page structure andcontent attribute data are used, along with consumption signal data, todetermine content consumption and/or performance metrics (306).

In various embodiments, content consumption may be determined based onone or more of the following:

-   -   Amount of content on the page: User engagement in a crowded        content environment may be determined based on factors like        viewable area, mouse interactions and content placement. In        various embodiments, user activities and indicators related to        user activity (i.e. mouse over/mouse hover), time on page, time        the article is viewed, and scroll depth and speed are taken into        consideration.    -   Page Layout: Cross platform and cross publisher display varies        by device and environment variables like layout and positioning.        Content consumption speed varies from device to device and        experience to experience. In various embodiments, such factors        are taken into consideration and engagement is weighed at least        in part based on content positioning and flow.    -   Article Viewability: Whereas in traditional environments a view        of any area of the page that is viewable is counted as a view;        in various embodiments other factors that may be better        indicators of actual engagement are taken into consideration.        For example, on load, an article may be in full view for a user        and that does not necessarily translate to that user actually        reading or engaging with that content. By taking into        consideration other actions and event listeners like mouse        movements, hover actions, scroll speed and interactions with the        surrounding environment, in various embodiments, more accurate        assumptions are created.    -   Slideshow/Image Galleries: In various embodiments, image        galleries and slideshow metrics are considered based on content        consumption and weighted engagement scores. Amongst these        parameters are the depth, time per image    -   User's screen resolution and screen size: Assumptions based on        screen size and resolution as well as the content being loaded,        pre-loaded or in full view are weighted by device. Mobile        devices have less real estate for diversions and provide a more        focused reading pattern. Nevertheless, it allows for less        content to be displayed to the user and requires more        interaction form said user to consume the content.    -   Scroll speed: As a measure of engagement, scroll speed may be        used in various embodiments to make determinations of user        attention and actual engagement. A user scrolling through an        area too fast can be assumed to be a disinterested user and as a        consequence the weight of the engagement metrics may be reduced.        In some embodiments, scroll speed-based parameters and metrics        may be used to increase the weight of user interactions that are        typical of average content consumption and engaged user behavior        and/or to diminish user interactions that may reflect the user        may only have skimmed the content.    -   Time: Time on article is not always an indicator of time engaged        on article. There are complexities to how time factors into        engagement. These complexities are based on the        multi-dimensional nature of the content and elements on the        page. An article with interactive elements that are being        interacted with like a photo slider cannot be matched in        engagement with an article with no elements that is not being        interacted with simply because both articles drove the same time        on site.

In various embodiments, metrics such as those described above may beweighted to assign more or less value to the different measures based onthe factors that affect them. A specific metric, like time spent on aphoto item, may be treated differently in a mobile device than on adesktop where the viewable area is in most cases larger. In variousembodiments, while the metrics being used to create the assumptions maybe persistent, their value is weighted based on factors affecting theenvironment. This provides further clarification and a more realisticview into engagement.

FIG. 4 is a flow chart illustrating an embodiment of a process todetermine content attributes related to measuring content consumption.In various embodiments, the process of FIG. 4 may be used to implementstep 302 of the process of FIG. 3. In the example shown, for eachcontent page the page source code (e.g., HTML or other markup languagepage code) is parsed to determine the content page's structure and toidentify content assets comprising the page, such as images, sections,tables, and other identifiable portions of content (402). For eachcontent asset comprising the page and/or a portion thereof, one or moreconsumption related attributes are determined (404). For example,depending on the content asset type, one or more of a main subject(e.g., of an image, text, etc.) may be extracted, and/or measuresindicative of how long it may take a typical user to fully consume theasset may be determined and stored (e.g., complexity, reading level,subject, size, length, formatting, visual context, such as clutter ordistracting adjacent content, etc.) For each page, the content pagestructure, content asset, and consumption-related attribute data isstored (406), for later use in measuring content consumption andperformance, as disclosed herein.

FIG. 5 is a flow chart illustrating an embodiment of a process todetermine content attributes. In various embodiments, the process ofFIG. 5 may be used to implement step 404 of the process of FIG. 4. Inthe example shown, for each content asset comprising a page, a contentasset type of the content asset is determined (502). For example, acontent asset may be classified as belong to one or more content assettypes, such as image, formatted text (e.g., paragraphs), bulleted text,table, product feature table, side-by-side product comparison table,etc. A content asset type-appropriate analysis is applied to determineconsumption related attributes of the content asset (504). For example,an image may be processed by retrieving and evaluating tags or othermetadata associated with the image; image processing may be performed todetermine a subject of the image (e.g., human face, infant, animal,etc.); social network signal data, such as “likes” may be processed,etc. Natural language and/or other techniques may be applied to text.Paragraph or other formatting (e.g., heading) information may bedetermined processed in the case of formatted text. A number of itemsmay be considered in the case of bulleted or otherwise formatted lists.Etc. A normalized “complexity” (or other measure indicating a normalizedand/or relative amount of value to assign to a content asset) isdetermined for each content asset comprising the page (506). Thenormalized value may take into consideration content attributes of thecontent asset, such as linguistic or other complexity, enhanced ordiscounted by such factors as placement within the content page, etc.

FIG. 6 is a flow chart illustrating an embodiment of a process toinstrument content pages to measure content consumption. In variousembodiments, the process of FIG. 6 may be used to implement step 304 ofthe process of FIG. 3. In the example shown, a code snippet to beincluded, e.g., by publishers, in content pages to cause consumptionsignal information to be gathered and reported is prepared (602). Thecode snippet is provided to content page publishers to install on theirsites to enable user engagement to be tracked, e.g., mouse movement,user time viewing page or portion thereof, which article they areviewing, etc. (604).

In some embodiments, consumption signal information may instead and/orin addition be requested from a dynamic content management system and/ora content ad server, and which content is requested and/or provided maybe tracked.

In some embodiments, content consumption may be tracked by using codeinstalled on publisher sites to measure the amount of content on thesite, understand what article a given user is reading, and how much ofthe article they've read.

In some embodiments, the code may also track user screen size, scrollposition, and such to know what is being consumed and for how long.

FIG. 7 is a flow chart illustrating an embodiment of a process tomeasure and report content consumption. In various embodiments, theprocess of FIG. 7 may be used to implement step 306 of the process ofFIG. 3. In the example shown, consumption signal data is received, e.g.,from pages downloaded by one or more users (702). The data may beaggregated and/or organized by content page, web site, user, etc.Content page structure and consumption-related attribute data (e.g.,content asset type, complexity, content value, etc.) is used, inconjunction with the received consumption signal data to compute contentconsumption and/or performance metrics (704), For example, a contentconsumption percentage or other measure of content consumption may becomputed for individual users, groups of user, on average across users,etc. Performance may be measure by determining by content asset,percentage completion of consumption of content comprising a page, etc.,correlated with post-consumption behavior data, such as which usersclicked through to read more detailed content, made a purchase, etc.Content consumption and/or performance reports are generated andprovided to recipients, such as publishers, advertisers, etc. (706).

In various embodiments, content consumption is measured across sites,devices, users, media, channel, etc. The challenge of measuring contentconsumption when distributed across sites includes but is not limited tothe fact that user engagement may be desired to be measured across anycombination of websites, digital platforms, mobile devices, set topboxes, gaming platforms, streaming devices, embedded software and/oroperating systems. In various embodiments, computed content consumptionmetrics are normalized across sites, etc. to enable consumption to bemeasured and compared across sites.

The complexities associated with the effective measurement of contentare emphasized in some embodiments by the diversity of devices and userconfigurations (screen size for example) through which the content isconsumed.

For a given instance of a user engaging with a given article (or othercontent), elements of the article that are in view on the user's screenmay be monitored. Each person's screen may vary in size, viewable areaand device. In various embodiments, how much time the user spent on eachviewable section may be recorded. A scroll speed and dwell time persection may be taken into consideration to determine if the personconsumed that section of the article.

In some embodiments, indicia of non-engagement may be measured and takeninto consideration. For example, if a full article is in view on thescreen, even if the user met the time requirement to consume thecontent, it may assume that they were not engaged if there is no mousemovement after the page is loaded and if the browser window or tab thatthe content loaded in was not in-view. It may also be assumed that theuser abandoned the page if too much time was spent on the page. If theuser scrolls to the bottom of the article but scrolls too quicklythrough a section of the content, the engagement for that section wouldnot be counted or mostly not counted as the user would have scrolled tooquickly to consume that portion of the article.

FIG. 8 is a diagram illustrating an example of a content consumption“map” display in an embodiment of a content consumption measurementsystem. In various embodiments, a content consumption map such as map802 in the example shown may be generated by a content consumptionmeasurement system, such as content consumption and performance analysisserver 122 of FIG. 1. In the example shown, content consumption map 802includes for each of a plurality of content assets 804, 806, 808, and810 a corresponding display area in which an asset identification datais displayed (e.g., “Asset 1: image”) indicating a content assetidentifier and content asset type are indicated. For each content asset,a corresponding normalized content consumption metric value isdisplayed. For example, in the example shown map 802 shows contentcompletion metric values of 20%, 10%, 40%, and 30%, respectively, foreach of content assets 804, 806, 808, and 810.

In various embodiment, a content consumption map display may include foreach content asset, in some embodiments including both sponsored (e.g.,advertising) and non-sponsored content assets, an indication of anexpected amount of time (and/or other indicia of consumption) for theaverage user (or some specific relevant user or set of users) to consumefully the content comprising that content asset. In some embodiments, apercentage completion as shown in the content completion map display mayindicate for each content asset a percentage completion, on average, ofthat content asset, e.g., based at least in part on the expected time(and/or other indicia) required to consume the asset as compared toactual observed time across one or more users. In some embodiments, thecomputation based on actual time viewing an asset may be “normalized” orotherwise adjusted based on factors such as placement within the page,whether scrolling was required to view the content, and/or contextualand/or qualitative factors deemed to increase or decrease thesignificance, effectiveness, and/or value of time spent viewing a givenasset.

In various embodiments, a content consumption map such as map 802 may begenerated for each of a plurality of sets of users. For example, in someembodiments, a separate content consumption map such as map 802 may begenerate for each of a plurality of demographic groups, e.g., age,gender, geographic area, etc., enabling content consumption behavior tobe compared across such groups. Such comparison may reveal, for example,that certain content assets are more effective than others in reaching agiven target demographic. If that target demographic is of particularinterest, existing content pages and/or future pages may be updatedand/or created to include and/or feature more prominently content assetsthat have been observed to be particularly more likely to be consumed byviewers in that demographic group, as compared to others.

In some embodiments, color or other visual attributes may be used toindicate the relative percentage consumption of content assetscomprising a page. For example, content assets with higher (normalized)content consumption may be shown in a darker or more intense color thanassets having lower computed (normalized) consumption. In someembodiments, the content consumption value for an asset may be weighted(increased) to reflect the observed performance of the content asset(e.g., a high percentage of those users who consumed 50% or more of thecontent asset clicked through to more detailed content); pre- orpost-consumption behavior (e.g., a relatively high proportion of usersdirected their attention away from but then returned to the contentbefore performing some desired behavior, such as clicking through torelated content); or other measures of the relative substantive content,value, and/or performance of the asset. In some embodiments, observedconsumption may be discounted to adjust for consumption that may not befully indicative of content performance. For example, image content mayattract users' attention for reasons not necessarily related to contentperformance, such as an image of a cute baby, a puppy, etc. In someembodiments, a consumption metric value may be increased to reflect alevel of user commitment indicated by consumption of an asset, such asby scrolling to the bottom of a page as displayed, etc.

FIG. 9 is a flow chart illustrating an embodiment of a process tomeasure content consumption based at least in part on individual userattributes. In various embodiments, the process of FIG. 9 may beimplemented by a content consumption and performance analysis system,such as content consumption and performance analysis server 122 ofFIG. 1. In the example shown, individual content consumption userattributes and/or behavior are determined, observed, learned, etc.(902). For example, scroll rates, time spent on content assets ofvarious types prior to indicia of completion, etc. may be observed andcompared to average consumption rates for assets of the same type. Userprofiles may be maintained for each user (904). User content consumptionattributes as stored in such profiles may be used to adjust consumptionsignal data and/or consumption metrics determined based thereon toreflect individual user attributes (906).

In various embodiments, a baseline is determined based on how much timeit takes an average person to consume each article based on the contentassets that an article contains. Individual user profile and/or observeduser behavior may be used to adjust based on whether the user consumescontent of the relevant type (e.g., by asset) at a slower or faster ratethan average.

Other considerations: international content consumption speeds may varyfrom region to region. In some embodiments, normalized consumptionmetrics may be adjusted to reflect such differences. For example, ifregional differences are such that an item of content may be expected torequire 20% more time to be consumed in Region A than in Region B, thento be assigned a same consumption value a user in Region A may need tohave been engaged with the same content for 20% longer than acorresponding user in Region B.

Content consumption speeds may vary per device (such as a mobile devicevs a desktop computer). The methodology in various embodiments adjustsconsumption time assumptions based on the device being used by the userto view the content.

In various embodiments, the methodology takes into consideration otherfactors that make up a comprehensive picture of user engagement. Acomplex mix of engagement metrics, patterns, consumption speeds,devices, screens, layouts, view-ability and signals may be considered.In various embodiments, one or more of the foregoing list ofconsiderations may be used to track engagement based on the environment,activity, technology and capabilities which affect performance and inturn engagement.

FIG. 10 is a flow chart illustrating an embodiment of a process toadjust content consumption measurement based on observed user behavior.In various embodiments, the process of FIG. 10 may be implemented by acontent consumption and performance analysis system, such as contentconsumption and performance analysis server 122 of FIG. 1. In theexample shown, for each content page, content consumption by contentasset, segment, section, etc. is observed and/or measured in acontent-aware manner, i.e., to reflect content complexity, asset type,etc., as disclosed herein (1002). Statistics are gathered regardingpre-consumption, content consumption, and/or post-consumption patternsand actions (1004). For example, each user's actions prior to engagewith the subject content may be observed, e.g., how each user arrived atthe page. Post-consumption patterns may include returning to furtherconsume an asset after having navigated away from it and/or engaging insome desired behavior, such as clicking through to related content.Content consumption and/or performance computations may be adjusted toreflect observed content effectiveness (1006). For example, contentoriginally determined to comprise a first percentage of the relativecontent value of a page may be increased relative to other assetscomprising the page to reflect performance of that asset.

In various embodiments, a content consumption metric as disclosed hereinmay be used for other media such as video where completion rates areused as an indicator of performance. For example, the number of users or% of exposed users that finished at least 25%, 50%, 75%, and consumedall of the content is reported for video campaigns. Prior to thetechniques disclosed herein, this metric has not been available forarticles that may include other forms of content—with video, the overalllength of the video and the amount of time a user spends watching thevideo or the frame of video that is reached while the video is viewableon the screen can be used to determine these completion rates. With anarticle comprised of various content assets, there is no overall lengththat can be easily determined and time alone cannot be used to measurecompletion since the entirety of an article may not fit on a user'sscreen. In various embodiments, techniques disclosed herein are used tocreate a multi-dimensional measure of content consumption that is moreaccurate and in line with user behavior and engagement. In variousembodiments, a unified “normalized” metric that can easily be used tocompare the success of very different type of contents being consumed invarious forms is provided.

FIG. 11 is a flow chart illustrating an embodiment of a process tomeasure and report content consumption across content sources and types.In various embodiments, the process of FIG. 11 may be implemented by acontent consumption and performance analysis system, such as contentconsumption and performance analysis server 122 of FIG. 1. In theexample shown, content consumption data is tracked for a user or set ofusers across content pages, sites, media, devices, delivery channels,and/or other domains (1102). For example, a user's consumption ofrelated content across web pages, related television ads, streamingmedia ads, etc., may be tracked, normalized to reflect platform/channeldifferences, and aggregated to determine a level of content consumptionfor the user. Performance may be determined based on observations acrossdomains (1104).

In various embodiments, the challenge of measuring content consumptionwhen distributed across sites includes but is not limited to the factthat user engagement may be desired to be measured across anycombination of websites, digital platforms, mobile devices, set topboxes, gaming platforms, streaming devices, embedded software and/oroperating systems.

FIG. 12 is a flow chart illustrating an embodiment of a process tomeasure consumption based on content value. In various embodiments, theprocess of FIG. 12 may be implemented by a content consumption andperformance analysis system, such as content consumption and performanceanalysis server 122 of FIG. 1. In the example shown, for each contentsection and/or asset comprising a page a percentage (or otherwiseexpressed proportion) of page content value to be ascribed to thatsection/assert is determined (1202). For example, natural languagetechniques, taxonomies, semantic models, etc. may be used to determine acontent value for each asset, and the respective content values may becompared to determine a distribution of content value for the page. Forexample, raw content values may be compared to determine a percentagecontent value for each asset, the percentages adding up to 100% of thecontent value of the page.

For each user who views the page, the user's interaction with thepage/content is monitored to determine the content sections (assets) theuser has consumed, and the compute percentages by section/asset are usedto determine how much of the content the user consumed (1204).

For example, if under the metrics being used ½ the content of an articleis considered to be included in the article's text and the ½ in a singlephotograph, a user who based on metrics and factors disclosed herein isdetermined to have viewed the photo actively long enough to beconsidered to have fully consumed it and who read ⅓ the text would beconsidered to have consumed ⅔ of the content of the article (½+⅓*½=⅔).

Computed consumption metrics may be aggregated, analyzed, and reported(1206), e.g., by user, aggregated by user type or attribute, etc.(1206).

In various examples described herein the term “article” has been used torefer to a collection of content assets, but the techniques disclosedherein can be used to determine completion rates of content thattypically would not be called an article, including without limitationother collections of different types of content, a slide show withcaptions, a “listicle” or an article primarily comprised of images oranimated images, a map, or a single image.

In various embodiments, measuring content consumption at scale inmetrics that are familiar to traditional market research, e.g., bycomputed a normalized content consumption metric aggregated across allcontent comprising an article or other unit of content (e.g., % ofcontent consumed across all types of content comprising an article) maybe used to understand the true performance of content as bothstand-alone and syndicated assets. A brands perception, influence andrelevancy in context and at scale enable publishers and/or advertisersto better understand whitespace, opportunity, pivots and rationales. Invarious embodiments, techniques disclosed herein may generate metricsthat align to how advertisers are accustomed to measuring campaignperformance at scale.

In various embodiments, a unified measuring unit that enables attentionand/or content consumption to be compared readily across differentexecutions is provided.

Although the foregoing embodiments have been described in some detailfor purposes of clarity of understanding, the invention is not limitedto the details provided. There are many alternative ways of implementingthe invention. The disclosed embodiments are illustrative and notrestrictive.

What is claimed is:
 1. A content consumption measurement system,comprising: a communication interface; and a processor coupled to thecommunication interface and configured to: receive via the communicationinterface content consumption signal data gathered by a plurality ofclients across a plurality of domains, the content consumption signaldata reflecting for at least a subset of content pages user engagementby content asset comprising a content page; use the received contentconsumption signal data and content attribute data associated with eachrespective content page to compute for each content page a contentconsumption metric reflecting an amount of content determined to havebeen consumed, wherein the content consumption metric is normalizedbased on a domain associated with the content asset; and generate anddisplay for each of one or more content pages comprising said at least asubset of content pages, a content consumption display indicating foreach of the plurality of content assets comprising the content page, anormalized content performance metric value computed for that contentasset.
 2. The system of claim 1, wherein the content attribute datacomprises content page structure data.
 3. The system of claim 1, whereinthe content attribute data comprises content asset type data.
 4. Thesystem of claim 1, wherein the content attribute data is associated withan amount of time required to consume a content asset of the pluralityof content assets.
 5. The system of claim 4, wherein the contentattribute data is determined programmatically by analyzing content datacomprising the content asset.
 6. The system of claim 4, wherein thecontent attribute data is determined based at least in part on a contentasset type of the content asset.
 7. The system of claim 1, wherein theplurality of domains comprise at least content pages, sites, digitalplatforms, mobile devices, set top boxes, gaming platforms, streamingdevices, embedded software, operating systems, or delivery channels. 8.The system of claim 1, further comprising determining programmatically,for each content asset comprising a content page, said correspondingproportion of content value of the content page that has been determinedto be embodied in that content asset.
 9. The system of claim 1, whereinsaid clients are configured by instrumentation code included in saidcontent pages to gather and report said consumption signal data.
 10. Thesystem of claim 1, wherein said content attribute data reflects for eachof a plurality of content assets comprising a page a correspondingcontent value attribute and wherein said content value attributes areweighted to reflect programmatically determined content attributes ofeach respective content asset.
 11. The system of claim 1, wherein theprocessor is further configured to aggregate content consumption dataacross content pages to determine a normalized content consumptionmetric value.
 12. The system of claim 11, wherein the normalized contentconsumption metric value is indicates a percentage completion or amountcompletion for the content asset.
 13. The system of claim 1, wherein thecontent consumption display is generated for a plurality of demographicgroups.
 14. The system of claim 1, wherein the processor is furtherconfigured to determine and report a content performance metric based atleast in part on said computed content consumption metrics and indiciaof effectiveness of such consumption.
 15. A method to measure contentconsumption, comprising: receiving at a content consumption measurementsystem, via a communication interface, content consumption signal datagathered by a plurality of clients across a plurality of domains, thecontent consumption signal data reflecting for at least a subset ofcontent pages user engagement by content asset comprising a contentpage; using the received content consumption signal data and contentattribute data associated with each respective content page to computefor each content page a content consumption metric reflecting an amountof content determined to have been consumed, wherein the contentconsumption metric is normalized based on a domain associated with thecontent asset; and generating and display for each of one or morecontent pages comprising said at least a subset of content pages, acontent consumption display indicating for each of the plurality ofcontent assets comprising the content page, a normalized contentperformance metric value computed for that content asset.
 16. The methodof claim 15, wherein the content attribute data is associated with anamount of time required to consume a content asset of the plurality ofcontent assets.
 17. The method of claim 16, wherein the contentattribute data is determined programmatically by analyzing content datacomprising the content asset.
 18. A computer program product to measurecontent consumption, the computer program product being embodied in anon-transitory computer readable storage medium and comprising computerinstructions for: receiving at a content consumption measurement system,via a communication interface, content consumption signal data gatheredby a plurality of clients across a plurality of domains, the contentconsumption signal data reflecting for at least a subset of contentpages user engagement by content asset comprising a content page; usingthe received content consumption signal data and content attribute dataassociated with each respective content page to compute for each contentpage a content consumption metric reflecting an amount of contentdetermined to have been consumed, wherein the content consumption metricis normalized based on a domain associated with the content asset; andgenerating and display for each of one or more content pages comprisingsaid at least a subset of content pages, a content consumption displayindicating for each of the plurality of content assets comprising thecontent page, a normalized content performance metric value computed forthat content asset.
 19. The computer program product of claim 18,wherein the content attribute data is associated with an amount of timerequired to consume a content asset of the plurality of content assets.20. The computer program product of claim 19, wherein the contentattribute data is determined programmatically by analyzing content datacomprising the content asset.